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ABSTRACT 
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program development, are aimed at improving currerit practic- 
^assisting in the establishment of new progra|m: practices. Th 
six sections deals with the current state of l| the art^^how n- 
assessment and evaluation are Currently bein|g done, . The sec- 
section analyzes a series o£ issues as well |a2§ enabling and 
inhibiting factors in the mental health field and their rel 
to continuing education- In the third section the report de 
operational definitions for the terms needs| ass essment and 
for continuing education in mental health and describes the 
consequences of lack of definition. Section^ discusses and 
a series of models for needs assessment and evaluation for ■ 
education in mental health and their use in various setting 
fifth section suggests strategies for conducting evaluation 
final section presents recommendations foj^ needs assessment 
evaluat ion. CDocuments on the follc^wing a^spects 'of continui 
educatidn in mental health are also avai^lable: state level 
preparing and using mental health manpower in state age: 
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FOREWORD 



In late 1975' the Meritkl Health Program" of the Southern Regional 
Education Board received a grant (No . 1-T15~MH14098) from the Continuing 
Education Branch of the National Institute of Mental Health to strengthen a. 
continuing education in mental health throughout the 14 states of the 
SREB region. The project conducted a survey* of continuing education ac- 
tivities then undeirway in the mental, health professional schools, societies 
and agencies, both state and community, to ascertain the heeds and problems 
'which are being encountered. Responses showed that afeas of major concern 
were: needs assessment; evaluation; funding; fining sanction; credential- 
ing; relations of professional schools, societies and agencies; and continu- ' 
ing education for paraprof essionals andicpimnynity caregivers. 

The principal, method of investigatioiy^ri this project has been the : f 

utilization of task forces of knowledgeable persons to explore these issues 
in detail and to prepare guidelines which might be of use to those presently 
responsible for mental health continuing education's programs or those who ^ 
will assume positions where they; will develop such programs. ' . " 

, ■> ■ . ■ ' " ■ ' ' j * ■ ' ■ ■ . ^ ' - * ' ' " ■ . " 

We are grateful to the National Institute of Mental Health for the 
support of this project and to the members of . the- task force who helped 
develop these guidelines on "Needs Assessment and Evaluation. "4 A* special 
acknowledgment goes to Dr. Robert Roberts, Director, Research and Training^ 
Florida Department of Health and Rehabilitative -Services, whose original 
contributions will be included iii his later publications. Di?. Don Cordes, 
Associate Director, Continuing Education at Virginia Commonwealth itniversity 
at the time he was on the Task Force, has also made a major contribution as 
chairman. . 



Dr, Harold L. McPheeters, M^D, 
Director, Commission on Mental 

Health/Human Services • f 



Frances R. Todd, Project Director 
^Continuing Education in Mental 
Health in the South 

. \ ■ ■/ :. ' - • 
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INTRODUCTION 



> Witljih the last five years, changes in the ddrections of mental healthu 
and mental retardation programs have gl^ven' rise to new and. different needs 
in continuing education in these' fields. Institutional pro^r amis are being * 
de-emphasized and community programs are expanding, . Departments af nursing, 
psychiatry, social work! and psychology aire being •replaced by geographic and 

^ . • ■ ■ ' ' ■ ' ' ' ■ . ' ^ ' ■ ■■ ■■-V 

problem-oriented units. New kinds of coirnaunity programs — group homes, half-. ' - 
.way houses y hot l:|.'nes and the like— -are proliferating . ^ 

' ' ' ; ' ■ ■ , ■■' ^. \ ^ ' ' > , . ■> , 

' Current legislation and class action suits . are resulting in: requirements 
Jor- more adequate treatment, ed;ucaticn and rehabilitation of released patients. 

There areVstrong pressures from federal and state governments for accountability, 

' • ' A'- ^ ■ ■ ■ ■ ■ . ■■■■ ■ ■ 

peer review and utilization review. Just to ikeep current demands more continuing 

education in the administrative and procedural aspects of mental health |^bgr^- 

rning* Another need, the development of techniques for using interdisciplinary ^ 

[continuing education as a management tool, will help implement new program 

directions and sharpen the effectiveness of program operations, 

A first step in 'arriving at guidelines for the implementation of effectlyiB 
mental health programming is a considerat;ion of "Needs Assessment and Evalua- 
tion." JThese words along with "Accountability" have become a part of the language 
of all the behavioral sciences. They are now a "must" in all planning and 
-implementation. * . 



^ •■■ . , ■ " > ■■ ■ / ■ ■ ■ ' • 

'• Aside from present acceptapce of the concept that needs assessment 
*and evaluation are essential in any mental Health program endeavor, the grow- 

Ing, emphasis on stanAards, credentialing, and accountability require -clarifi- 

: ■/ ■ ■ ■ ■ ' " ."■ ■ '• . • ■ ' •. . •• ".' 

cation of the how, when and why of the assessment of need and Vevaliiation of 

outcomes in mental h-eal^th continuing education. • It; is cleai; that to obtain 

administrative arid financial support for continuing educatiori, programs must 

present evidence of effectiveness. Current public policy requires account^ 

ability in nearly all huirian service programs, and survivarl depends upon a 

• * . I- ■ ' , ■ ■ 

competent analysis of requirements and evaluation of results, . 

Continuing education programs dealing with the administrative and 
programming aspects of mental health services are on the increase in the South, 

They have sprung up both from consumer demand and as- a result of the value 

\ ■ . ■ . ' ■ ' •■ ' \ ' ■ '■' 

judgments of tTiose planning the. content. Programs , may have been planned on the 
basis of a fairly adequate need^ assessment or on a purely whimsical basis; 
The Task Force has looked at the questions concerning need^ assessment and 
evaluation in mental health continuing education in the South, has examined 
evidence of' effectiveness or ineffectiveness of existing types of programs and 
has recommended guidelines for the use of those initiating projects or those 

pl^£ 

This report will not b^concefned with the technology of needs assessment 

• ■< ' ■ " ■• ' - ■ ■ , ■ " , ' - ' ■ ■ 

arid evaluation only, as there are established source^ for securing training 

■' '.■ " ' ■ . ' • ■ ■ ■ ' ■ ■ ■ • - ■ 

and information in' this phase of the subject* The guidelines will, however, 
examine such factors as the^ context in which certain technologies Should be 
employed, the. opportunitites and limitations : of different methodologies ajid the 



reassessing existing pl^s . ■ 



I • ■ ■ . • . • ^^^^-^ " • 

jrationale for assessing needs and evCLWtXTnife. Recommendations are aimed at 

the improvement of current practice)^ and assistance in the establishment of 

sound procedures in new programs and are intended for personnel at various 

levels and at different stages of program develgpment. 

' D EFINITIONS OF .TERMS \ • • / 

While the terras need'^assessment and evaluation are deeply embedded in 



the current; eciucational jargon, much confusion still exists regarding their 
meanin'g and appii(!iation._/^S^is report is concerned with a comprehensive 
discussion of needs assessment and- evaluation applied to continuing education in 
mental' health. . , \ ■■ \ . . 



The definition of continuing education for this report is one generated 
by the Southertx Regional Educatidn Board (BarlMLer in this project. . That defini- 
tion is: "Continuirig eduqatipn iis a system of learning experiences aimed at 
"improving, modifying or updating one's knowledge, skills or values in areas of 
prbfessyonal or occy^i^'flar pra^tice^. 

There is ; frequent d'^bate as to whether or not staff development and con- 
*tinuing edVjcation are separate, distinct entities. While there are distinct 
characteristics :dii;ferentiating the two' in the area of continuing education for 
professional praatitiohers , for the purposes of this document an institution 
or agency can^treat them the same in regard , to needs assessment and evaluation, 
it is important to note, however, that, d^ending on the intent, of the educa- 
tional activities' and the target audience involved, ver;y different strategies . 
and' models^ will achieve maximum impact. These differences are explored in 



sections fpufr and five of this report, 
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ASPECTS OF NEEDS ASSESSMENT. AND, EVALUATION TO BE COVERED 



Atk), EVALUATION TO BE COVERED !^;f I'^M-^ 



These guidelines deal with five distinct areas* The>Titst.-^^ is an 
analysis of the current state of the art how needs assessment and evalua- 
tion are currently being done- The secondvarea analyzes a series of issues 
as well as enabling and inhibiting factors in the mental health field and 
their relationship to continuing education. Third, the repott develops 
operational definitions for the terms needs assessment and evaluation for 
continuing education in mental health and describes the consequences of lack 

of definition. The fourth area discusses and analyzes a series of models for 

■ * ' ' . ■■-*■' ■ . 

heeds as;?essment and evaluation for continuing education in mental health and 

their use in various settings. The fifth and final area deals, with strategies 
of conductiaag evaluation. 



L 



; ERIC : 



STATE OF THE ART 



PURPOSE OF CONTlMiNG EDUCATION 

U.. ' ' ' ■' 



The ultimate'ipi^rpose of continuing education in mental health is to 
assure that mental, illness and mental retardation services are suffcient, 
both inVuality and quantity, to meet the total, i^eds of the population being 
served. Thus, continuing education must be client- or patient-centered if it 
is to achieve maximum , impact . It is, therefore, implied that to be effective, 
patient-centeffed continuing education must begin with patient-centered needs 
assessment and conclude with patient-centered evaluation. . 

ROLE OF CONTINUING EDUCATION 

Ttfe extent to which a continjiiing education pi!x)gram can achieve this ^ 

purpose depends upon its role within the institution or agency. If continuing 

. ' ' ■ . . '■ ■ " ^ ■ ■ . ■ - ' ' ■■ 

education is closely integrated with the' organizational purpose, it. must by 

■ * ' '' \ ■ ■ ■ . . , ' . ^ . . 

definition relate directly to patient or client services. To have a^ effect 

■ : ^ K ■ 

on patient or client services, agency-based continuing education must parti- 

cipate in the agency's overall patient- or client-centered needs assessment 
and evaluation activities. If on the other hand, continuing education within 
the agency, institution, or organization is perceived as a mechanism to deliver 
exotic new information in mental health, rather ^ than as a tool to provide, 
appropriate client services, a far less demanding needs assessment and evalua- 
tion tool may be used. In this case, needs assessment jnay consist of asking 



tho5e persons attending the program whether or not the information provided 
was on target with their ^perceived needs, v 

' It appears, however, that in agency practice, continuing education must 
be rel.ated to the delivery of serviced to clients* There are increasing demands 
by society for accouritabiiity in patient or client services as well as in 
education. Therefore, the continuing educators must be, able to, relate their 
activities directly to patient or client services and must be able to diemonstrate 
and document high quality patient services*as a result of those activities. 



CURRENT ACTIVITIES \ ^ . ; .. 

A recent survey. by the Southern Regional Education Board (April , 1976) 
illustrates these shif tjLngVt)ri0rities . The survey was completed by some 64 
agencies, societies, and inst^^i^itions who provide continuing education in 
mental health in the Sautheastern United ^tates. Results of the survey indi- 
cate that an attempt is nea^rly always made to determine what consumers think 
■they want, while * there is , little actual measurement of services delivered. In 
terms of evaluation/ nearly everyone evaluates trainee satisfaction by asking 
£he trainees whether they thought the pr^xigram was successful in meeting Jiheir 
needis. A significantly lesser number evaluate changes in learning. An even 
smaller number evaluate changes in* job behatrior, and almost none evaluate 
the impact on client services. The inconsistency between what most mental 
health professionals hold as "an ideal and what actually takes place in needs 
assessment and evaluation is evident. 1 . 
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FACTORS AFFECTING ME N TAL HEALTH CO NTINUING EDUCATION 

' -* . ~ ■ — T " . • - • . / 

/ 



NiBlther ipental health servic^s^or continuing^ education 'In mental health 

.\ ' ■ ' • . .. ' / ■"■ _ .. 

exist or function in a vacuum; they both function in an environment which - 

significantly affects the success of their endeavors. To be ^an ef fective >con^ 

■ ' ■ . ■ • /'• " . ■ . . 

tinning educator in mental health, one must understand the environment in 

which mental health services are provided. This section deals w'ith issues in 

mental health which have the potential for affecting continuihig^ education, . 

including how one performs needs assessment and evaluation. 

t^HiLosoprii CAL Conflicts / ■ : ^ ' / . ■ . . • : \ 



jthery gire- a: series of philosophical conflicts' within the mental health 
system which must be dealt with^ by mental' health professionals dLn* institutions. 



agencies, atid organizations before continuing^education can be carried out 



successfully/ An exaftple of such a cdnfljic^t 'is whether an -instiiEtition ^hduid - 
provide quaiity custodial care fi^r^its lori^'-term patients or wliether a vigorous . 
community support strategy shoulld be eimployed/ to get the pat;ients back iritp the 
community as soon as possible. Another' conflict exists regarding which is the 
appropriate treatment model to use in any particular setting: '^a medical modely 
,a social taodel, a behavioral model or an educational model of mental health 
activities. A third philosjskphical conflict cpncerns how mental health -ishould 
be involved in primary, secondary, and tertiary prevention and how such involve- 
ments sWuld be programiqed. > ^ 



. If .an ins titiit ion V organization,, or agency does not clearly specify where 
it $rtands on these and other, philosophical issues, it is difficult to dfevelop. ; 
'suecessful continuing education. It thfenbecopies necessary for continuing 
Educators to *4etermine ih some^ysteinatic fashibYi the philosophies and goals 
of their particular institutions. " ' ' » * : 

POTENTIAL CONSUlffiRS • • - ^ • : ■ ^ ; ; : . 

Different. giToups have different kinds of cbntinuihg education .ne^ds,* The 
term "consumer" has different )uses as it relat[©;s jfco mental health. Primarily 

' ^ • ■ -■ ■ ' ■ ■■ ^' ■ ' ■ ^ ': 'V::.r^i . " ■ • • 

. the consumer is the^ traditional client or, patletlt, the user ofVmental 'health r 

'-/.'* ■»' ' ' ' . ' , ' ' ' " . . '' 'j', ^ ■ - ' , ■ '■ ■ " . . • * 

services^; Continuing education for/) the client ox family might Ke thou^t ^of 

as consumer education. In another sense the consumes of contini^^Tig education 

■■■ ■ ' ' . . ■ . ' ■■ - . "■ 'V'::::'''^' - ■. 

are /the traditional providers o£ mental health services* * ^'Providers" 'may be 

thought of as all thpse mental health prof essionals arid/ par apr of esslonals who , 

V :^ ■ ' . V ' '/■'^■■- ■■. ^.^ -, . v-V.r.''- ■■: . ■ 

provide mental' health services to' clients; Workshops or seminars • are the most 

common forms of mental health continuing education for providers .* ^ 

Continuing educators, too, have continuing education needs iir mental • 

' • ■ 'I" 'c^ ^' - ' ^^ . 

health. Their needs center around liew edlicationai techniques, tools ^ and . 
methodologies to facilitate learning among the providers and clients. For this 
group, continuing education has most often consisted of experiehtial learning" 

activities iri' the processes and techniques of adult learning .^^ ~ . ' 

* [»■' . ' ', ' ' ' ...*■' ' . ' ' . . ■ *'. ■ 

■ ^ ■ ■■ ■■ ■ ■ - ■ ■ ■•■ ' ■ ■ . " ■ ■ ■, . v' ■ 

A fourth- group with continuing education needs may be described as the 

"community caregivers." These include^ policemen, school teachers, and other , 

'professionals in the community who deal with mental health problems, but are • 



not geniarally thought of as mental health professionals. Their continuing 
education needs obviously relate 'to their, specific professions and the situa- • 
tions in which they interact w4;#'cli^nts having mental heklth problems. 

Each of these various groups ^must be involved in needs assessment and 
evaluation processes related to their own continuing education. FurtheiyniQre, 

. - . . " . . . • . ■ . , ^ ■ 

.within agencies there are many levels of personnel, and all must be reached 

" ■ ' ■ '.A ' ' ' - ■- .■ ■ ' ■ ■ ■- ■.' ■- 

with continuing education to improve quality of Service delivery. 

■• "•». ' .' ■ . . ■ ■ ■ 

INFLUENCES ON CONTINUING EDUCATION IN MENTAL HEALTH 

''■^^S^^^i^^ ^^^tal health agencies, there are several factors which may > affect 

■ * . ■ > ■ ^ ' ■ .. . 

the success of continuing education needs assessment and evaluation activities.. 

.' •• ^ V,- ■ ■ > " ■ ■ . ■. ■ 

These factors have the potential of being either inhibitors. or enablers. 

Following are a serieis of factors /which can influence continuing education 

needs assessment .and evaluation in all institutions and organizations providing 

mental health selrvices. 



Relicensure a nd Certification 
. An important issue to be considered is the increasingly commoTi requirement 
of continuing education for relicensure and recertif ieation of mential health 

professionals. Continuing educators must be aware of. the specif ic requirements 

' ' . ^ ■ /• ■ ■'■ * ■ " 

for recertif ieation, ^ relicensure or renewal of professional society membershipji - 

■ ' ' ■ * - 

for the mental vhealtji professions they serve and, whenever possible, must provid 
appropriate educational experiences. It is important ta note ^howeyer, that 
merely having all of the mental health professionals in the agency or institu- 

t ion certified or licensed does not. necessarily guarantee, high quality mental 

■ • ^' * ■■■ ■' '.■ • . . ■ • • . • . ■ . 

health services. ' ^ . 

* • V " - ' « ■ \ ■ ■ ■ ■ ■ : ■ 



TSie personnel «3^steTO affects mei^^ and, 
therefore, mental health cbntiriuitig education needis assessiaeht anci evaluat^^ 
Often the role which an individual may be asked tq play within an institution-' 
is defined more in terras of a generic jobvdes crip t ion developed by: the state 
personnel office than by specif ic functions; seriously 'affect how 

one goes about performing fieeds assessment and" evaluation for continuing 

education of the agenky as a whole. 

* . ' • . • - . ■ ■ 

Reiiabursement Requirements - ' \ • . 

Rersonnel systems nj^y also, facilitate continuing education by requiring 
employee participation to be eligible for promotioris or pay raises.' Another 

■ ^1 ; . ..,».'". 

influence is the reimbursement system for third party payments. In building 
staffing patterns for mental health services, it is necessary to realii^e that 
what may seem' optimal in the eyes of the* mental health agency planner may; 
conflict with what third party payers will allow in such a situation* In 
such a case, inappropriate third party payment patterns inhibit appropriate 
continuing education. 

- . ■ . '3 

' . " ■ ' - . ■ ■ . ' ■ ■ ' . ■. • 

Professional "Turf" ' 



Another factor common to all agencies,' institutions, arid organizations 
may be described^as "turf struggles'' between professions and units. It is. ■ 

• ■ ■ ■ s ■ ' ' : ■• . ■ ■ . ' ' ■ 

essential that continuing educators understand the basis for these turf 

■ ' '. ■ . ' . 

struggles within the agency or institution and. 'work with them to^the best 
advantage of the program. Sometimes these struggles result from a "blurring 
of roles," and continuing educatioti programs can be designed to meet the 
needs of all^.professlons rather than jus^ one or two. , . 



C6i;itlnuity of Continuing Education ' 
. Continuity of the continuing education activities affects the potential 

success of needs assessment and evaluation, ' All' too of teif^ ..a department of . 
continuing education sponsors a number of continuing education activities 

' whicli have little relationship tp each ©ther resulting in a sporadic, dis- ' 

. ► * ■ ' , ■ ■ . ■ ■ ^ ' ' 

jointed series of learning experiences. Continuing educators then must change 

■ ' •• . * " ' " . ■ "■ ■ ■ • ■ -" ■ 

from random, one— shot programs to an ongoing series of interrelated learning 

*■'•'■ . . ■ . ' ' " ■. ' ■' • " ■ 

activities produced in a systematic sequence. This also implies that continu-; 
"*.■■■ ■ ■ ' . ' ' ' • ' 

ing education is an integral part of the agency's program and manpower 

development fort . - 

Standards Requiring Continuing Education 
/ Mental health continuing education is facilitated by -the : standards of 

various regulatory agencies which accredit health care delivery facilities and 
programs;: The Joint Commission piTS^e Accreditation of Hospitals (JCAH) 
requires evidence ftf continuing education for staff within an institution 

or agency. ^ ' ^ . . 

■ ' ■' ■ ' •■ • . • • " ^ . 

Also, the Professional Standards Review Organizations (PSROs) require 
patient care evaluations/ These evaluations may be an excellent place to 
'begin needs assessment. Understanding and relating to the roles of these - 
agencies will greatly facilitatp the task ofT a^mental |health continuing educator, 

t Community Expectations 

Another factor influencing continuing educatibn is the lay public ^s image 
of mental health servicesJand its professionals. The ^continuing educator 
* must biecomie aware of the community expectations within which the agency, 

, •■' ■ . ' • -11- ■ ■ . ■ . ' , ^ 
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institution'or organization exists. This implies that the continuing 

educator has access to and is involved in comprehensive community assessment 

\ .' ■ ■ - . ' \ ■ . ■ ■ ' • " 

and analysis. It may also indicate a need to provide mental health continuing 

k. . . •■. . 'V .'^ ■ • . ■ . ■ ., ■ 

education to the lay public residing in the community to improve understanding- 
of the mental health services. 

■ ' • ■ . ' , . ■; ■ - . ■ . • » . 

Funding for Continuing Education • - 
Funding is a major influence in mental health continuing education. ^ 
Within each agency continuing education must be provided a /sufficient funding 
priority so that the program does not have to "sell" its offerings on the basis' 
of popular appeal rather than true need. A coalition of staff mental health 
professionals might be helpful in establishing a high priority for funding 
^cpntinulng education within the agency. ' 

~ ■ » . ■ 

■ ■.- "^^ . ' . . 

. Expertise of the Continuing Educator ■ ^ • 

An enabling factop in continuing" education needs assessment and evaluation 

is likely to be found in the expertise of the continuing educator. Too often 

.- ' . V ^ ■ • ' ■ . ■ ■ . ' ' . . ^ . ■ , . 

mentlil hiealth professionals without appropriate training are thrust into the 
role of^ontinuing educator. It is essential that continuing educators recog- 
nize their deficiencies and develop ways tp alleviate those deficiencies. 

While-all these issues and fattors do not affect all "institutions all the 

" ' ■ "■ ■ ■ ■ ' ■ ' ' - ' ' \ ' ' ■ 

time, the'*' degree to which continuing educators recognize and deal with them 
will bear directly upon their effectiveness. The foregoing examples of in- 

* " n . - ■ _ . , ■ 

fluences illustrate factors affecting the whole of mental health continuing.. 

education as well as the needs assessment \ah<:^Bpiluat ion part ions. They are 

area? to. which continuing educators should bJrever alert in developing programs, 



CONCEPTS ^ 



Needs assessment and evaluation very often involve the same methodologies, 

but at different times and for different purposes. In needs assessment, judg- 

* . ■ 

ments arfe made to determine educational or other activities necessary to assure 

. . . ■ ' ■ . . ' ^ ' ' * ■ . ■ 

optimal service delivery. In evaluation, effects are examined, as a result of 
a particular planned activity. It is important to understand how' needs assess- 
meat and evaluation interrelate in order to deicde what types of tools to . 
employ. Jdeally, needs ass^^ssment and evaluation tools are compatible with* 
one another. It would be inappropriate, for example, to determine "needs via 
a paper and pencil test, then to evaluate for effectiveness by examining patient 
records. One would have no way of knowing whether any real change had occurred. 

Often a needs assessment will^uncover deficiencies which do not relate to 
mental health content or process but, rather, result from breakdowns In logistic 
or communication .mechanisms . These deficiencies cannot be corrected with an 
education program, because lack of knowledge' is not the real problem. In such . 
instances, a mechanism must be employed to bring these deficiencies to the 
attention of the appropriate agency administrators so that they can be alle- 
viated. Jf the mechanism does not currently exist in the agency or institution, 
it is the responsibility of the continuing educator to alert the appropriate 
Authorities and to hielp d^elop a mechanism for use in these situations* 



To define needs afssessment, a .series o:^ concepts must /be dealt w^h. 

Needs assessment is a process. 'It must be systematic , and it must involve the 

> ' ".• / • . -• ■ ' . ' ' . ck' '"^ _ , ■ ■ ' ■ ■ ' ,' . . /■ ■/ .• - • • - 

people whose, needs are being assessed. - It is not sq|n.ethin'g t-liat is done to 
someone, rather 'it is something that a continuing educator does with sbfceone. 
T^he process should also be objective and quaA|ifial:)le^^ that 
there is data collection and construction of "an approj^ base. The 

needs, assessment must also' be comprehensive', dealing w±<^h(l>6th the present and 

"■■ ■ / : . ■ ' -V' ; ■"V^- \ ■:, • " • - ■ 

the future. Following is a^def Inition of ne^'^ds aissessmeift which attempts to 
embody these conqepts. ' .f:'^'^'^^ . ' 



Needs -assesismerit is a syst.emat^^V yc^^ 

and quantifiable pro tes'^^ Gj^T^-t^^ whose 
needs are being assessed for. thje: purpose of planning and / 
implementing act iyit£es:Iiije.c the delivery 

of optimal mentals health ..sic^vit!^^'" 



As ^with. needs "asses sme^'njt, .there i^e^^cep^s which must be embodied in a 

definition of evaluatltSn in ^^raejatal^ too, is a process. It 

■. . ■ * ^^'^ - -'"^^ ■ '^fe ' ■ 

entails setting a standard «(which^ ir4'8hii; fee ^^ <^^^ a desired state) 

f.,^- / _....v ' -'^^ ■. ' -'.'^'^ 

and comparing that ^staudardvto"^^^^^^^^^ Also involved is analyzing 

or making a value judgmeVtj^agl 'to whether qr not is. sufficiently 

like the desired gtatC:^ '<:T^^ these, processes must be comprehensive 

(that is, they mus^^e^siite^boih intended .and uninte^^eU outcomes) , and they 



must .be adaptabl!^ '(that, is , "^ble to function in the pa^^icular settings in 
which the learning^^-acpivities are carried out) . , 31ie^ fC>^4-5l-ow definition 



^embodifeS:V.J,he above"^^ concepts : 



lEyaluatipn^i^n mental health continuing education is an 
.a^^^ valid, reliable, and: adaptable 

'\ ' . pTCocessV^tilized to 'determine the extent to which a 



particular learning, experience or set of experiences * . 

has produced a desired outcome. 

Similar processes are involved in botl;i needs assessment and evaluation. 
Subjective judgments should be kept to a minimuio and reliance placed on objec- 
tiVe data, > whether the needs assessment or evaluation is ^simple or complex. 
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STRATEGIES FOR CONDUCTING' NEEDS AS SgSSMRNT - ' 



. ■ Needsr assessment has been -definecl as a systematic, 
tive and (iuantifiable process involving;those whose needs ^are beii|'a^^^^ 
This section will describe how to conduct such a need^ assessmeW^^iid^ g^^^ .: 
illustrative models, - ; ; ^ . ^ 



The choice of. a tnodel for needs" assessment a^^ 
.. . the size of the agency or institution, program goals, the natiie 'of the/ : ' . . 
. organization, ^the pbrtion of the program evaluated,: personnel; and ^^t^^^^ exp^rr :, 
tise, funds available, and other factors. / : ■ ; -"' ^ ^/ : 1^ 

I . ■ , . . ■■ ■ , ' , ■.' .!.■'"•"• 

^ ■ A basic framework for conducting ,needs assessment in mental health con- 
. tinuing education, is ;given in Figure 1.. Needs exist' at; three'^levels: client - 
needs for services, £rovide_r needs for professional education and 'trkining, and 

• institutional * -needs for service. delivery. Two general^^bints need to.bemade, 
. First, the scheme is consistent with ~the premise that the.,most effective ' 

continuing education activity is built upon a :client-c'entered, iieeds; assessment. ' 
Second, two major sources of needs assessment data exist at ekch level -objel- 

• tive measures ,, such as demographic, survey, an^ "treatment statistics; and . - 

^ ...:.£^^ mandate^^^-^rpe^ pressuTes. 

These two sources of data interact, and no needs assessment is complete unless 
both are taken into account. ' 

■ *Institutional will be utilized throughout this discussion' and can be 
^ taken- to mean agency or organizational as well|f 

^ -17- 
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A needs assessment profile for clients draws upon such indices as 

^ ■ • . ^- ' ■ ■ 

^incidence and. prevalence data, treatment utilizationstatistics , and various 

■ • ■ '•■ ,, r 

socib-economic indicators, along with perceived needs in Jthe form of laws 
and regulations, professional opinion, and other data. These two major 
^sources together define the scope t>r extent of client needs , 

. Provider needs , in turn, are defined, first, by objective meagures of 
' Identified discrepancies between the services required by clients and the 
professional services provided by staff , and second, by perceived needs in ^ 
the form of personal priorities, institutional demands for cc^tlnuing educatiofi 
arid the lacl^. ^ . 

Finally, institutional needs for service' delivery are -of two types: needs 
for professional skills training as identified at Level II/, and more general 
indices of organizational effectiveness — staff sat isf act ion, turnover rates, 
organizational communication patterns, and so forth, -This last category 
suggests that the scope of the continuing education practitioner may well 
extend beyond the traditional arenas of service delive 

\ This model is designed to provide a 'conceptual rramework and di^monstrate 
how the various sources of data interact to produce /a final comprehensive 
needs assessment for an institution or agency. It /also demonstrates that 
there are "three distinct sources from which "to gatftier "data relating to con- 
tinning education needs in mental health. 

• DEFINING CLIENT NEEDS 

The first data source is the individual clients which, in the collective 

■ . ■ ' ' *^ ' ' 

: : ■ ■ -19- 



form, are the community as a whole. To operate a successful continuing 
education program it : is highly desirable to have an objective data base. The 
data base for the community consists, in part of demographic and socio-cultural 



characteristics of the area to be served. -This information can b^ obtained 

• ' ■ ■ ' o , ■ , ' - . • ■ ' ^ ■ 

*for a nominal fee from the Bureau of the Census, U.S. Depart?menl: af Commerceii 



It also consists of epidemiological data available f rem Jthe Office of Biometry 
of the National Institute of Mental Health (NIMH) and vital statistics (births, \ 
deaths, incidence and prevalence of disease^ divorce, illegitimacy, etc.) ; * 
available for the community to be served. This, data can be obtained *from com- 
munity inventories. Assoc iat.ed types of data are utilization and referral -v^j 

' u ^ . . . . . ■• •■ .'1;^^, 

patterns. Both of these can be obtained from agency or clinic records in thef r;# 

target^area- ' / ^ V:t ^ 

. _ • • , t,. 

Objective data concerning individual clients can be obtained by perform-* 

' ( ' . ■ ' *. " ' ' ■ ■ ' • '\ 

ing a Service Delivery Analysis . This requires the continuing educator to , ^ 

» ' ■ ■ . •. • ■ . - ' ■ 

examine the service delivery for specif ic categories of /clients and thqjti com- 

^ ■ ' ' / ' " ' . '-^ ■ / ' ■ • ■ 

pare the actual client out comeS/ to the pre-^set definitions. , . ^ 

Subjective data, referred to as perceived client needs in Figure 1, are 

- * " ' • ' • • ' '•" ■ / ■ • 

also esgential in determining client needs. An important source of this type 
of data is public opinion, . This oan be assessed by conducting polls and 
surveys as well as by analyzing media coverage of mental health. Another 
source is recent or pending legislation or court actions relating to mental 
health. Important/ and often neglected, are sources of data regarding 
" Community attitudes and values toward mental health. This data can be collected 
by employing attitude scales or value inventories as well as group proces^ , 
techniques, such as the Delphi/Delbecq. " ^ /' 

-20- ■ ' /. ■ 
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DEFINING PROVIDER NEEDS - . . ^ 

Numerous data sources /.are available to help assess the objective needs of 
v: providers relating to service delivery and professional and career advancement. 
The first, and most commonly used, method is the, knowledge teat ; In its 

simplest form it consists oS true-false, multiple choice or short answer ' 

y ■■ 

questions. In newer and more- sophisticated circles, it consists bf client- ^ 

■ ' ' ■ . ' - ■ ^ . . . ■ ■ • , . ■ ■ ., ■ ' " ' •• " - '.-^ ■ 

management prpblems or written simulations. Some professions have self-asse3S- 

/*; . • • : ■ ■: : -.v^ ■ ■ - . , ;.• . ' '-' ■ , ■ ■, . . •■ ' 

$ ment examinations^ of this kind. Another important method is the psychomotor 

skill test' ^which assesses physical skills. " A third and increasingly popular 

• •• f ■ • ■ , / • ^ ' • . ' ■ . - 

and meaningful method is the, analysis of the quality of client care . This can 

be accomplid^d by becoming involved in Ipcal PSRO activities as- well as by ^ 

examining the degree of adherence to external standards, such as those of the 

. Joint Com^Sissioni of the Accreditation of Hospitai^'^^StKl^ the Social Security 

Administration, 

Data on the career needs of providers can be gathered by examining 
credentialing and relicensure data generated for mental health professionals 
serving the institution. The various professional groups may. be consulted to 
identify new knowldcjge and technology which should be included ;Ln continuing 
, education programs. Another method of assessing provider's career needs is 
analysis of the merit system requirements and potential career ladders" within 
the agency. Mental health providers should hav^^ the opportunity to move 'into ' 
more challenging positions, and continuing education provides one mechanism 
for persons to learn new skills to ascend the ladders. A final, and widely 
used method of gathering data is the procurinjg of felt or perceived ar^as of 
concern or deficiency from the providexs themselves. This can be done in small 

' • *■ ' . ■ ' . . ^ ' "21^ ' ^ , 



group process sessions employing a Delpfii or Delbecq technique or with 
questi'^nnaires or otheir instruments • ^ ^ 

DEFINING INSTITUTIONAL IJEEDS - •/ 

^Data helpful in determining/institutional. continuing education needs comes 

'"•■■^ ' . ■ ' \. ' ■ . ■■ • ' ■ ■ ■■ ' \- ■-' • ' 

from a variety of sources. External" data may come in the form of direct ^ * 

■ • , ■ > . •/ . . ■ ■ 

legislative mandates > specif jLc consumer pressure ^ "accreditation requirements . " 

or union demands > Each of these is readily -available to the continuing educa- 
tor and should be utilized. o 

Internal sources of data are inhquse political vpressureis , alterations of 



the goals or purposes, of the institution, changes in staff numbers or composi- 
tion , changes in employee expectations , and problem' areas perceived by the - 
supervisory and administrative staff . The data can be acquired by the continu- 
ing educator through consultation with the supervisors and administrators of' 

■ ' ' ' ' ... ' . ' . - .- 

the agency, , ^ ^ 

■ ' •» ■" • . • ■ 

Two other sources of data are pr;of essional literature reports on the 
development of new service delivery techniques and the perceptions of continuing 
educators themselves Ultimately it is the persons in charge of continuing 
education who mus-t synthesize and pull together the various data into a meaning- 
j^ul whole. Because these persons are privy to such , an extensiye array of d^ta, 

their' judgment is an important consideration (see Figure 2) ♦ 

"* • " • ■ . ■ ■ ' 

SYNTHESIS OF DATA . 

Once data has been gathered from each of the three sources, it must be 
synthesized, analyzed, and put into a useful form. Figure 2 demonstrates the 



« 

.• • • '■' ' 
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quasi- Interdependent relationship of the three data sources and suggests a 
Strategy for indexing tiie datai so that needs can be ^ven priority. 

The paradigm states that c^ata may indicate certain staff training needs - 
which are exclusive of other interests.; other data may indicate needs^^common 
to two interests; and still other data may indicate needs that are common to 
all th.'cee interests. ' i /" . - / . \ \- - 

iThls offers ,the possibility of seven strategic target areas, which can ... 
be ranked ordered » for continuing education^ For example, the data which 
fall into area "C*' indicate a training need which will address only the needs 
'^of staff. The data which fall into area "A"' indicate -a training need which 
wilL address" only the unmet needs of qXients, and so on. An item in area 

. ■ ... , ■ ■ ■ . ■ ■■' ■ t . . ■ 

"G" wo^ld be a first priority training activity since it^ddresses the needs 
^of all three interest group^s. . 

A^ an example, assume tb:e case qf ^n Institution with an unusually high 

■ , - .fs^' • ■ . ■ " 

mortality rate among a partlciijLar client group. The question becomes, "What 
interest group is affected by these data and wliere is that dfnformatidn placed 
in Figure 2?" The clients? Yes, of course; they-are the ones who -are dying, , 
The staff? Yes, of course; they are subject to professional liability • !rhe 

- . ' ' a ■ ■ ■ . . 

institution? Yes, of course; it is subject tq^ public and/or legislative - 
censuret "Where are thp data placed?" ""Data are placed in ai^a "G", because 
the needed training will ameliorate an unmet need of each of the tfii^e interest 

groups: life for the clientte^ protection* for the staff and public estefem for 

• ,1 ■ ■ . ' ■ ' ■ ■ •> . . 

the institution, ^ \ * . ^ 
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The needs assessment process, depicted in Figure 3 offers a fqrmalized ^ 
mechanism to answer systematically the three needs assessment questions ^i^ a 
wide range of settings. The description of the desired state' should consist 
of a set of measurable and achievable objectives, such as meeting national,'' 
state or local standards, meeting professional standards of^variolis di'sciplineSj 
meeting Certification criteria, and attaining a level of achieyement, that ,is 
>et by the agency itself^ This i^^^bas4cally a leadership function with. appro- 
priate input from various units, iiacluding the continuing educator* The 
description of the existing state is a middle level management function because 
this leyer of staff has^ the most immediate access to the detailed data required 
to generate a set of statements descriptive of the. current ''conditions.. 

Completing these first two. steps answers the first quiestion— Is there an 
■ ' ■ , ■ ■ , '•' ■« . • ' ■■*■... • • . - 

unmet need for any. of the three interest groups? The difference between the 
■K'v-;- ' ■ ■ r ■ ■ 

existing and dei^ired states ' calls for inputs from many diverse persons relevant 

to "how and why the unmet need' exists; and what might be done to resolve it . 

Completing the - third step provides the answer to the second ques tion-- -Can un- 

met need be resolved by continuing educ'atiQn activity? ' 

^ ^- T^ of the fourth step is ai matter of describing, designing, and 

^developing a set of activities , events , and resources which will , when properly 
implemented,' lead from the existing to the 'desired state. This will answer 
the r third question — -What must be done to resolve the unmet need? The process 
"yields a set of staff training needs which may be- "special", or "commpn^* needs 
as illustrated in Figure 3. 



FIGURE 3 




ERIC 



DESCRIBE, DESIGN AND;^ 

DEVELOP A SET , OF. 
ACTIVITIES WHICH WISl ■ 
LEAD FROM THE EXIST IlfG 
T^THE DESIRED STATE 



DESCRIBE IDEAL 
OR DESIRED STATE 


A 




DESCRIBE EXISTING 
' '/ STATE " • ' 

* . I'M' 




r ■ 


DESCRIBE DIFFERENCE 
BETWEEN IDEAL AND 
EXISTING STATE ./ 








ONGOING PROCESS 
OF FORMATIVE AND 
SUMMATIVE EVALUATION 
AND REVISION, 



ZD 



The complete needs assessment process generates a set of prescriptive 
statements relat.ed to what must be done to resolve the unmet need, rather than 
a descriptive statement about wha;t is wrong. Function fdur is primarily a 
design function and requires the input of staff members who are skilled in . 
design, development, planning and evaluation procedures, ipe., instructional 
designers and program planners. 

A compreherfsive staff Js^raining needs assessment is a critical element in 
the continuing educator ' s long-range planning strategy. /Without -it i educa- 
tional activity is prone to become no more than a series of random efforts 
which fail to flow into an integrated pattern of events, -and which may or; may 
not^ address the needs of any of the interest groups. 



. . STRATEGIES FOR CONDUCTING EVALUATION - ; 

* \ : ' [ ~ 

V ' ■ - . ■ . ; 

^ • • r . • • . 

Earlier in this report evaluation in mental health continuing education / 
was defined as an analytic, comprehensive/ objective, and adaptable process 
used to determine the extent to which a particular learning experiende or , 

, V''. ' 

set of experiences has produced a desired result . Several methodological 
implications are embodied in this .definition. To be ''analytic" an' evaluation 
must deterraipe not only the success or :tailure of a program but also must 
prpbe and explore possible explanations for the outcome. To be "comprehensive, ' 

an evaluation should assess program effects in a variety o£ areas including, 
"P ^ 
• ' - ' ■ * ■ • ' ' ■ ■ ■ • ^ •■ 

but not limited to, intended and unintended impacts ^upon services- to clients 

or patients* * 

■ ' ' ' * 

To be "objective" an evaluation must rely upon multiple sources of data : 
' encompassing'" muitiple points of view. And finally, an -'adaptable"' evaluation ^ 
is one which Is designed to, meet evaluation needs of a specific problem or 
setting, not one which is put together strictly from an academic modeli In 
short, good program evaluation In mental liealth continuing educatipn should 
be designed so that it serves specif ic program decisions . The extent to 
which evaluation is able to "do that is the extent to which it pays for itself 
in terms of more effective and efficient continuing education programs . 
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Given tliese /premises^^ that thei most ' important step in 

'eyaluat^rng..co:n^ is determining appropriate iBvaluatiag 

critaria ; ' What . ndeds> to be known about this program, and who needs to know , 
it? if' Reyelc^ntr,aud^^^ should be identified, i.e. , those\^ 

persons qt ^groups yhoh^Ve some interest in or policy impact on the continTiiug 

ducatiori program. Most often, relevant audiences will include faculty, or~. 
ganizational or institutional administrators, external funders or certifiers 
of profes'si'onal skills,- and actual or potential program participants and their 

supervisors. Although all have an interest in continuing education programs, 

• ■' • ' . ' . ■ ' ■ ' ' - ' ■ ■ • . . . 

• the specific evaluation criteria likely to be important to each group will ' ' 
vary. For example, faculty kre apt to be most interested in data which will 
lead to instructional improvement, while external audiences are more likely .to 
focus upon particular skills learned by progra.m participants . Since continu- 
ing education practitioners are in some way accountable to each, of these groups, 
they are well advised to check periodically on their expectations • Examples 

. of Useful^ criteria-generating questions are, "What sorts of evidence would con- 
vince you that this progrjam (or.: series, of programs) is . effective?" Or, "If this 
ptogram were, successful , ^ what would it be doing? What sorts of effects would 
^t have?" V : ' ; \ ' - • ' / 

Any criteria-generating process is likely to result in rather lengthy 
■ ' ' ' . ■'■»■■ ' ' . . ' . 

lists of important criteria*. However, the criteria are likely to fall into 

: "■ ■ • ; • ■ ' ■ '■ , ' , ^, ■ 

certain cleaV and distinct levels ox categories.. Three major critetia levels 
are: planning ,' process , and;^^ outcome . .Representative criterion questions ate 
listed for each level. 



LEVEL I - PLANNING CRITERIA FOR A CONTINUING EDUCATION PROGRAM 

Is the plan an integral part of a larger plan? 

Does, this plai^conform to local, state and /or national 

rstandards? ^ /' - ' 

■ . • . • . .. ■ \. ^' . 

Have data from previoufe . continuing education programs been • 

used in planning this program? . ^ .• 

' ' ^ Were appropriate outcomes identified? 

• Is f there a clear congruence between the stated objectives and 
. , the identified need?, : 

Are tha. objectives attainable within reasonable time 
constraints? ' 

' . Have the pertinent leaders Accepted the plans? ^ • ^. 

Has the consumer accepted the plan? ^ 

Are resources available to meet the stated objectives? , 

, ^ ■ ■ 

Has a formal evaluation mechaniism been built in? 

'Planning' criteria generally r_elate to the congruence between- identified 
needs and program objectives, as well as;. overall program feasibility, given 
available resources and /potential participants. 

LEVEL II - PROCESS CRITERIA FOR A CONTINUING EDUCATION PROGRAM 

. . / Did the expected ^umber aind type of providers arid^^c 

partitipate? . ' 

Were methods. used'appropriatiB; to the content? 

Was the learning environment appropriate to the objectives? 
. (Instructors and Participants). 

Did the program . proceed as planned? . If riot , why? 

What critical incidents occurred to facilitate or inhibit 
the process? , n 



How did the participants view (feel about) the learning 
experience? 

Were the learhing objectives met? 
/Were the expectations of the learners met? 
Can the participants put into practice what they learned? 

Process criteria generally reflect the success of a particular continuing, 
education ievent,. from both the consumers' and practitioners ' point of view. 

\. LEVEL III ^ OUTCOME CRITERIA FOR A CONTINUING EDUCATION PROGRAM 

Did the participants put into practice what they learned?. 

Did the program accomplish what it was expected to accomplish? 
If so, what factors made i.t su^c^ssfyl^^If not, why not? 

'. - ■ v". ■ . * ' ' ' ' ■ ■ ' ■"'■■' ■/ ? ■ "■ ■ ••' •■■ ' ' ■ 

What were the unanticipated outcomes?J^-'^^^^>'^ ^ [ ■" 



What were the long-range developmenxai (catalytic) outcomes? 

Did the program have, an impact on the identified needs? 

^» . ' ■ ■ ■ ' ■ - 

a. Short term , 
b • Long term 

'Given. the above, was the program worth the time and money?" 

As may be seen from these sample' questio^, program . outcomes may be Of . 
sevferal kinds: Direct effects (question 1) kfe anticipated, short-term 
results; sid^e effects (question 2) are often imexpected (e.g., personal or pre 

■■. ■ ■■■ ■■;.',■.? . . : - : : . ' 

fesslonal .contacts ma:de, incidental learning) catalytic effects (question 3) 



are. precursors to more long-term outcomes (e.^., new training needs identif ied<'. 



practitione'r credibility established for other Jiprograms) . Together , all three 

• ■ . ■ v" ' ■ ■ ■■■■■■ 

types of effects define a. program s overall impact . Often evaluation studies 
have focused upon direct effects — accomplishment of prespecified objectives — ; 

■ ■ ■ ■■ -32- 



to the exclusion of other effects which are often more important, A single 
program may not 'have impact upon identified client needs, but if it sets 

other ev^ts in motion, it may serve a very important catalytic role. 

<^ ■ " ■ ■ ■ . . " 

A:numhei: of different sources may be called upon to provide data relevant 

• ' • • '. ^ . . 

to criteria at all three levels. - : 

Program participants consitute a primary '^and commonly used resource for " 
evaluation data.* Consumers pf continuing education tend to have specific and 
realistic ideas pf what they wish to learn and are, therefore, usually good 
judges of program value. Further, decisions about, program revision and support 

tend to be based largely upon opinion and reputation, and. program consumers 

, , . . • • ' ■ ' • . 4?- 

are the major source of such opinion* ^ A' number of studies have shown, however i\r> 

■ ■■ ^. ' ' - ' ' . '"■ '■ ■■ ■ ■ * • ■' . ' ■ ■ - o • --y : ' A 

that consumer satisfaction often bears little relationship either to the amount? 

of learning or the extent. to which learned skills are applied ./'back home." | ' 

Therefore, while sa:tisf action may be a necessary condition fpr^ program success;" 



it is usually not sufficient . ^ 

Program f iles are another important source of data. Prior evaluations^ of 
similar programs, attendance patterns, participant characieris tics, and reasons, 
for attendance may' all provide helpful clues for the plapning of future prbgrams, 
as well as illuminating the probable "market 'V for the' ehtire continuing' educate 
effort itself. 

-V Data obtained from the participant ^ s professional setting are helpful bath 
in. documenting need arid in isolat/ing long-term program effects. Patient care 



audits, peer ^review; supervisor ^/-a tings, .structured observation, and ^interviews' 



•all represent u^ful data collection sources. The majoi: drawback to such 
methods is that they; tend to be both expensive and time-consuming if they have 
to be set up especially for the evaluation effort. For continuing ecucat ion 
practitioners faced with a large number of pro]>rains » such systematic follow- 
up may b^ unrealistic as a routine actitivty • 



In these cases, a sampling 



process of both participants and programs may be employed. 



^ Fina] 



Finally, there are regulatory s ources of 



evaluation data. As discussed 



in the earlier sections on needs assessment , regulatory bodies help determine 



the perceived needs for continuing education 
toring sucl# things as licensure and certif icat 
rfrquests by official agencies , continuing^ educ 



assess the extent to which perceived needs hav|e been met - and new ones 

• c • ' ■ 

generated, • , 

' In the Figure 4 -the three criterion levels have been crossed with the four 
ina.jor sources , of data. :The letter codes in efch cell represent appropriate 



Accordingly, by routinely moni- 
ion requijrBments and program 
ation practitioners are able to 



data collection techniques. Others, of cours 



e, may be added. 
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PQ== Participant Questionnaire 
KT= Knowledge .Test 
PT= Performance Test 
PP== Participant Product- ,. :; V/ 
ED= Prior Evaluation Datai ' 
~ PC= Participant Characteristics 
SR== Source of Referral 
IL= Implementation Log 
CI= Critical Incident Analysis 
FC= . Foliow-up Communication, 
solicited or unsolicited 



PR= Peer review, audits 

SuR= Supervisor Rating 

NAQ- Needs* Assessment _ Questionnaire 
CS== Case Study 
LE= Legislation 
CE= Certification Requirements 
PS= Professional Standards 
RE= Requests by Of ficiar Agencies 




RECOMMENDATIONS 



Strategies for needs assessment and evaluation of continuing education 
must take into consideratib^^he available personnel, o support and funding, 
and organizational commitment 

' t'' ■ ' ' . ■ . ' ^ 

At the outset, there should be some ^fcind of evaluation plan, no matter 
what the size or nature rof the agency or.'institution. The plan should be^ 
based on. defined needs , specific objectives, and meth9ds which ^re practical, 
and understood in advance. One of the problems in evaluating inental health 
i;>rograms today is that hospitais drid mental health centers have been, established 
without clearly deterTuined/needs . The present emphasis' on "deinstit:,utionali 

. zation'* and community support systems requires a new assessment of contiijuing 
education needs to" fulfill the changing mandate, of: the courts and the largexi^^jr 
society. There is a mandate far different apiirbaches to meeting the req^uire- 
meats of the. mentally impaireli , but how. does this translate int^ 

. services? Continuing educators should participate in the planning for the . 

service, programs and use this knowledge to ^plan the evaluation for the continuing 

"I ' ■ . 

education program. ^ --^ . 

Much programming is based on what groups or individuals historically felt 

-■. ' ' ■ / . ■ .*■ *■ ■ ■ ' I " ■ ... ' • ■■ ■ I 

was needed and what would work. . As a result, one becomes involved in assessing 
established systems which require an updated detertninat ion of present needs.. 
A mental health agency ar institution should plan an ^valuation for any prog?: am. 
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which is being moijified or initiated so that the evaluation plan is designed 
before the fact , . The same is tru6 olE continuing education programs. 

At the state level. Departments of Mental Health and Retardation usually ' 
have at least one experiienced eval^a tor, working at an office of Research and 
Statistics. or an office of 'Planning anii Evaluation. Such an evaluator may be 
a helpful consultant in assisting continuing educators to evaluate continuing 
education programs. , . 

.A ^ •. , ' . . , ', ' ' •• ■ ■ , * ■ ' J"* . ■ ■ 

> ' ' ' . ... . >■ . ' - ■ , . ' ' ' ; • " . 

../;.Ano~ther optjLon is to'^se consultants from other 'state or local agencies i f 
universities or private groups . On occasion an external evaluator . serves a 
useful purpose,* but regular needs assessment and. e^ in- 
ternal process with continuity/ follow-up, arid feedbacks ' 

A significant factor in needs assessment and evaluation is that of 
attitude. Many persons plead that human service wd'rk canno^S^life evaluated, 

that there- is not a specific product which, can be measured.^' it has 

" ■■ ; ^ / ' ^} ... V--" ^' ■ - ; . ■ ^ /A ' 

been e^ablished' that most^elements of a mental ^Ijealth continuing education . : 
program can be evaluated, and that there a^e gPaany facets; :to^ ^tt^ . eivaluation 

process. All must begin with the willingness of. the agenc^^ the: faculty and - 

• V, ' :: . . . ' ■ ' . ■ ■ ■■ 

the participant's to havfe the effectiveness of the various prt)grams assessed. 

" ' " ' , ■ -* . ' .■. ■ 

" Nee<}s. assessment or evaluation does not always have to be elaborate or 

complex* If • a total operation, cannot be analyzed at one time, a manageable 

•■'*■*"■' ' . ^ ■ ■-" ' • ' . ■. " ' ■ ■ ' ' . . ■' . " ■ . • .. 

portion 6f san^le^ of programs can be individually assessed or evaluated. The 

stage of development of an agency, personnel available, and funding determine 

the depth and breadth of the needs assessment and evaluation, M6st accrediting; 
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or ^uncjing sources have, several levels of acceptable compliance, and goals 

can be set for achieving higher levels as time goes on'. Whatever mechanisms 

•' ^ ' ■ ■ ■ . " 

are. eihployed,. advice can be secured from experts who can judge appropriate 

V ' ■ ' . • • . , • .. ' • ■ ' •■■ 

. methods for a given' sitbation. =- . r , - / 

Vniile needs assessment and evaluation begin with providing services to .. 

< clients, this is also the framework for continuing education^ One of the 

• ' ■ ' . . . ., ■ ''■'■■*'■.■■•. ■ . ' . ' .'^ ■ ' 

^ tools for bringing about program change is continuing education, Which must 

itself be evaluated - \ ; 

The complexity of a nee4s assessmeht or evaluation* is detei^bqed by the 
goals or objectives of the particular project or unit. A relatively s^^iple" 
needs assessment and evaluation, for example, might be employed in a ^pontinuing 

■ - , ■ ^ - r \ ^ ■ ■■ ' [. ■ . ■ ' ■ 

education experience for a limited group of smai^l town police. In this case,^ 

^ . ^ ^ .... . ' . " ' - — , . .-. - - . . 

the objective might be to learn p.rocedures for referring intoxicated persons 
to I Alcoholics Anonymous. * Needs could be determined by asking the police if 
this was a problem for them, how oft6n-,they encountered it and what they usfually 
did; After the training course they could be asked if 'they learned what they 

expected, how th^y would handle such cases after the training, and if they / . Iji. 

■ ■ - ■ ' . ■ " • . ' ' * . ' ' ■' ■ ' ' • 

■ ' .-■ ■ ' ^ . • ■ ■ . ' ■ "'■ ■ 

thought the experience waal. useful . They could also be asked to provide followup 

. and feedback on experiences subsequent to -the training.* \ ^ 

, . If the ^request was for a continuing education p^ro gram. -of one-hour weekly 
sessions' for .one :year, combined with consultation regarding actual cases f for 
a large group of p6Hce in. a big city,, moire, elaborate needs assessment anid *^ 
evaluation would be required. Data would be .collected showing tha incidence 
of certain kinds of case problems in the are^» the educatioh bac'kground ^. ■ 



of trainees, and ref^eVral statistics . of -1 training methods 

' wouid be defined; with pipi^e cpmple^ evaluatlpn ;proceddres , such as exaTnliiatlons, 
foil6w-up interviews ^nd/or surveys. Depending upon the goaLs of the program, , 
there tilignti ej^en bie a comparison study with a. similar group of police who had ^ 

■ . "■■ ■ ' ■ ■ I ■ ■ ' . : ■■ ■■. 

no t; Jiad - the tyaiiiing-t^)-^ — 
■'program* •' "'^ ■ ' .. 

At the other extreme of the spectuTnn,',6ne might have the i^egftbrisiblllty^ 
of determining the need for ahd.^ evaluating a Gprisultatlon and Education program . 
of a, large Mental Health Center. In this case 'bne wpuj^ use a nui^ber of ^ • ; 

•different methods arid instruments, some of which. have been.\gtandardized . 

i> ■ -^^^-4^^' \.\ \ . : ■ ■ 

■■ ■" to: ^ \ 

■ : At this time thi&* "State of 'the Art" in Needs Assessment and Evaluation of 

Continuing Education is that the need for accounting for and judging what 

^ practitioners are doing with clients and systems^ is recognized; the te^ihnology 

for doing this is emerging; and there is a need to continue to search for 



answers which will validate whether practice in mental health continuing edu-^ 
cation'is effective in t^rin&ing^^^a^^ change for\,the clients of the . 

system../ , ;■ ■ . ;■ ^ .• ■ ^ . 
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